Barebones Background for Markov Chains

0. Introductory Remarks. This collection which I refer to as “Barebones Background for Markov Chains" is really a set of notes for lectures I gave during the spring quarters of 2001 and 2003 on Markov chains, leading up to Markov Chain Monte Carlo. The prerequisite needed for this is a knowledge of some basic probability theory and some basic analysis. This is really not a basic course in Markov chains. My main purpose here was to begin with a minimum de…nition of a Markov chain with a countable state space and to proceed in as direct a manner as possible from this de…nition to the strong law of large numbers for Markov chains, i.e., the ergodic theorem. This prepared the way for me to introduce in a mathematically rigorous way the Hastings-Metropolis algorithm for Markov Chain Monte Carlo, concluding with the optimality theorem by Billera and Diaconis. The only kind of Markov chain that I needed to deal with was an irreducible chain with a countable state space. Thus these notes should not be construed as a basic course in Markov chains but simply a course in the background mathematics needed for a rigorous justi…cation of the Hastings-Metropolis algorithm. These are called notes for the simple reason that, other than statements of de…nitions and theorems and proofs of theorems, there is no expository prose included other than these introductory remarks.